Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.
translated by 谷歌翻译
High order structures (cavities and cliques) of the gene network of influenza A virus reveal tight associations among viruses during evolution and are key signals that indicate viral cross-species infection and cause pandemics. As indicators for sensing the dynamic changes of viral genes, these higher order structures have been the focus of attention in the field of virology. However, the size of the viral gene network is usually huge, and searching these structures in the networks introduces unacceptable delay. To mitigate this issue, in this paper, we propose a simple-yet-effective model named HyperSearch based on deep learning to search cavities in a computable complex network for influenza virus genetics. Extensive experiments conducted on a public influenza virus dataset demonstrate the effectiveness of HyperSearch over other advanced deep-learning methods without any elaborated model crafting. Moreover, HyperSearch can finish the search works in minutes while 0-1 programming takes days. Since the proposed method is simple and easy to be transferred to other complex networks, HyperSearch has the potential to facilitate the monitoring of dynamic changes in viral genes and help humans keep up with the pace of virus mutations.
translated by 谷歌翻译
古本(Guzheng)是一种具有多种演奏技巧的传统中国乐器。乐器演奏技术(IPT)在音乐表演中起着重要作用。但是,大多数现有的IPT检测作品显示出可变长度音频的效率低下,并且在概括方面没有保证,因为它们依靠单个声音库进行训练和测试。在这项研究中,我们建议使用可应用于可变长度音频的完全卷积网络提出了一个端到端的古兴游戏检测系统。由于每种古季的演奏技术都应用于音符,因此对专用的发作探测器进行了训练,可以将音频分为几个音符,并将其预测与框架IPT的预测融合在一起。在融合过程中,我们在每个音符内部添加IPT预测框架,并在每个音符中获得最高概率的IPT作为该注释的最终输出。我们创建了一个来自多个声音银行的名为GZ_ISOTECH的新数据集,并创建了Guzheng性能分析的现实世界录制。我们的方法在框架级准确性和80.76%的笔记级F1得分方面达到了87.97%,超过了现有的作品,这表明我们提出的方法在IPT检测中的有效性。
translated by 谷歌翻译
本文旨在探讨如何合成对其进行训练的现有视频脱毛模型的近距离模糊,可以很好地推广到现实世界中的模糊视频。近年来,基于深度学习的方法已在视频Deblurring任务上取得了希望的成功。但是,对现有合成数据集培训的模型仍然遭受了与现实世界中的模糊场景的概括问题。造成故障的因素仍然未知。因此,我们重新审视经典的模糊综合管道,并找出可能的原因,包括拍摄参数,模糊形成空间和图像信号处理器〜(ISP)。为了分析这些潜在因素的效果,我们首先收集一个超高帧速率(940 fps)原始视频数据集作为数据基础,以综合各种模糊。然后,我们提出了一种新颖的现实模糊合成管道,该管道通过利用模糊形成线索称为原始爆炸。通过大量实验,我们证明了在原始空间中的合成模糊并采用与现实世界测试数据相同的ISP可以有效消除合成数据的负面影响。此外,合成的模糊视频的拍摄参数,例如,曝光时间和框架速率在改善脱毛模型的性能中起着重要作用。令人印象深刻的是,与在现有合成模糊数据集中训练的训练的模型合成的模糊数据训练的模型可以获得超过5DB PSNR的增益。我们认为,新颖的现实合成管道和相应的原始视频数据集可以帮助社区轻松构建自定义的Blur数据集,以改善现实世界的视频DeBlurring性能,而不是费力地收集真实的数据对。
translated by 谷歌翻译
社会建议利用社会关系来增强建议的代表性学习。大多数社会推荐模型都将用户互动(协作领域)和社会关系(社会领域)的用户表示统一。但是,这种方法可能无法模拟用户在两个域中的异质行为模式,从而损害了用户表示的表现力。在这项工作中,为了解决这种局限性,我们为社会建议提出了一个新颖的截面对比度学习框架DCREC。更具体地说,我们建议从项目和社会域中学习分开的用户表示。此外,分离的对比度学习旨在在分散的用户表示之间进行社交建议之间的知识转移。各种现实世界数据集的全面实验证明了我们提出的模型的优势。
translated by 谷歌翻译
反事实是一种新兴的模型解释类型,最近引起了行业和学术界的大量关注。与传统的基于特征的解释(例如,归因)不同,反事实是一系列假设样本,可以将模型决策翻转而对查询的扰动最小。鉴于有效的反事实,人类能够在``假设的情况''的情况下进行推理,以便更好地理解模型决策边界。但是,释放反事实可能是有害的,因为它可能无意间泄漏敏感信息给对手,这给模型安全性和数据隐私带来了更高的风险。为了弥合差距,在本文中,我们提出了一个新颖的框架,以生成不同的私人反事实(DPC),而无需触摸已部署的模型或解释集,在该集合中注入了噪音以进行保护,同时保持反事实的解释作用。特别是,我们使用功能机制训练自动编码器来构建嘈杂的类原型,然后根据差异隐私的后处理免疫从潜在原型中得出DPC。进一步的评估证明了拟议框架的有效性,表明DPC可以成功缓解提取和推理攻击的风险。
translated by 谷歌翻译
边缘计算是一个将数据处理服务转移到生成数据的网络边缘的范式。尽管这样的架构提供了更快的处理和响应,但除其他好处外,它还提出了必须解决的关键安全问题和挑战。本文讨论了从硬件层到系统层的边缘网络体系结构出现的安全威胁和漏洞。我们进一步讨论了此类网络中的隐私和法规合规性挑战。最后,我们认为需要一种整体方法来分析边缘网络安全姿势,该姿势必须考虑每一层的知识。
translated by 谷歌翻译
Vision-Language预培训是一个新兴和快速发展的研究主题,将多模态知识从丰富的资源预训练任务转移到有限资源下游任务。与主要学习单个通用编码器的现有作品不同,我们提出了一种可训练的通用编码器 - 解码器网络(UNI-EDEN),以促进视觉语言感知(例如,视觉问题应答)和生成(例如,图像标题)。 UNI-EDEN是一种基于双流变换器的结构,由三个模块组成:对象和句子编码器,其单独了解每个模态的表示,以及通过模态交互能够实现多模态推理和句子的句子解码器。考虑到每个图像的语言表示可以跨越该层次结构的不同粒度,包括从简单到全面,个人标签,短语和自然句子,我们通过多粒愿景语言代理任务预先列车UNI-EDEN:屏蔽对象分类(MOC),蒙版区域短语生成(MRPG),图像句匹配(ISM)和屏蔽句生成(MSG)。以这种方式,UNI-EDEN赋予了多模态表示提取和语言建模的功率。广泛的实验证明了通过微调到四个视觉语言感知和发电下游任务来展示Uni-Eden的概括性。
translated by 谷歌翻译
Temporal sentence grounding (TSG) aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning. However, we argue that these methods have overlooked two indispensable issues: 1) Boundary-bias: The annotated target segment generally refers to two specific frames as corresponding start and end timestamps. The video downsampling process may lose these two frames and take the adjacent irrelevant frames as new boundaries. 2) Reasoning-bias: Such incorrect new boundary frames also lead to the reasoning bias during frame-query interaction, reducing the generalization ability of model. To alleviate above limitations, in this paper, we propose a novel Siamese Sampling and Reasoning Network (SSRN) for TSG, which introduces a siamese sampling mechanism to generate additional contextual frames to enrich and refine the new boundaries. Specifically, a reasoning strategy is developed to learn the inter-relationship among these frames and generate soft labels on boundaries for more accurate frame-query reasoning. Such mechanism is also able to supplement the absent consecutive visual semantics to the sampled sparse frames for fine-grained activity understanding. Extensive experiments demonstrate the effectiveness of SSRN on three challenging datasets.
translated by 谷歌翻译
Robust prediction of citywide traffic flows at different time periods plays a crucial role in intelligent transportation systems. While previous work has made great efforts to model spatio-temporal correlations, existing methods still suffer from two key limitations: i) Most models collectively predict all regions' flows without accounting for spatial heterogeneity, i.e., different regions may have skewed traffic flow distributions. ii) These models fail to capture the temporal heterogeneity induced by time-varying traffic patterns, as they typically model temporal correlations with a shared parameterized space for all time periods. To tackle these challenges, we propose a novel Spatio-Temporal Self-Supervised Learning (ST-SSL) traffic prediction framework which enhances the traffic pattern representations to be reflective of both spatial and temporal heterogeneity, with auxiliary self-supervised learning paradigms. Specifically, our ST-SSL is built over an integrated module with temporal and spatial convolutions for encoding the information across space and time. To achieve the adaptive spatio-temporal self-supervised learning, our ST-SSL first performs the adaptive augmentation over the traffic flow graph data at both attribute- and structure-levels. On top of the augmented traffic graph, two SSL auxiliary tasks are constructed to supplement the main traffic prediction task with spatial and temporal heterogeneity-aware augmentation. Experiments on four benchmark datasets demonstrate that ST-SSL consistently outperforms various state-of-the-art baselines. Since spatio-temporal heterogeneity widely exists in practical datasets, the proposed framework may also cast light on other spatial-temporal applications. Model implementation is available at https://github.com/Echo-Ji/ST-SSL.
translated by 谷歌翻译